The relevant prior art for auto-focusing with motion estimation estimates the motion of an object based only on distance information. Therefore, its estimation has been limited to the motion of objects which move along an optical axis. Japanese Patent Laid-Open No. H07-288732 concerns devices which store the shape of the object of interest, and tracks it using the shape as a template. However, these devices do not use the distance information of the object of interest, nor do they estimate the future motion of the object of interest. Therefore they can track and focus only very slow objects.